Devices showing improved resolution via signal modulations

ABSTRACT

Techniques for displaying an input image in improved perceived resolution are described. In one aspect, a circuit is designed to include a set of memory cells, a horizontal decoder and a vertical decoder. An input image is received at an interface to the memory, the input is expanded into two separate frames in the memory, where the size of each of the two frames is identical to that of the input image. Image data in at least one of the two frames are modulated in amplitude and/or in phase. The first and second frames are then read out or displayed alternatively at twice the refresh rate originally set for the input image to achieve the perceived resolution.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of co-pending U.S. application Ser.No. 16/389,912, filed on Apr. 19, 2019, now U.S. Pat. No. 10,546,521issued on Jan. 28, 2020.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention generally relates to the area of display devicesand more particularly relates to method and architecture of modulatingboth amplitude and phase simultaneously within a pixel or within asingle array of pixels, method and apparatus for creating embossedmicrostructures or specific alignment cells to control the amplitudemodulation and the phase modulation, method and apparatus forcontrolling the optical characteristics of liquid crystals using photoalignment, nano-imprinting lithograph (NIL) and E-beam, and method andapparatus for controlling voltage being applied to a liquid crystallayer to realize the modulations of both amplitude and phase atsubstantively the same time.

Description of the Related Art

In a computing world, a display usually means two different things, ashowing device or a presentation. A showing device or a display deviceis an output mechanism that shows text and often graphic images to userswhile the outcome from such a display device is a display. The meaningof a display is well understood to those skilled in the art given acontext. Depending on application, a display can be realized on adisplay device using a cathode ray tube (CRT), liquid crystal display(LCD), light-emitting diode, gas plasma, or other image projectiontechnology (e.g., front or back projection, and holography).

A display is usually considered to include a screen or a projectionmedium (e.g., a surface or a 3D space) and supporting electronics thatproduce the information for display on the screen. One of the importantcomponents in a display is a device, sometime referred to as an imagingdevice, to form images to be displayed or projected on the display. Anexample of the device is a spatial light modulator (SLM). It is anobject that imposes some form of spatially varying modulation on a beamof light. A simple example is an overhead projector transparency.

Usually, an SLM modulates the intensity of the incident light. However,it is also possible to produce devices that modulate the phase of thelight or both the intensity and the phase simultaneously. SLMs are usedextensively in holographic data storage setups to encode informationinto an incident light in exactly the same way as a transparency doesfor an overhead projector. They can also be used as part of aholographic display technology.

Depending on implementation, images can be created on an SLMelectronically or optically, hence electrically addressed spatial lightmodulator (EASLM) and optically addressed spatial light modulator(OASLM). This current disclosure is directed to an EASLM. As its nameimplies, images on an electrically addressed spatial light modulator(EASLM) are created and changed electronically, as in most electronicdisplays. An example of an EASLM is the Digital Micromirror Device orDMD at the heart of DLP displays or Liquid crystal on silicon (LCoS orLCOS) using ferroelectric liquid crystals (FLCoS) or nematic liquidcrystals (electrically controlled birefringence effect).

Currently most daily watched displays and their related technologies,either at home or office, such as old CRT, LCD, OLED, LED billboard,camera, office projector, home theatre projector or even digital cinemaprojector in movie theatre, are only related to amplitude modulatedintensity of images which are direct or indirect reproduced images asamplitude modulated display. As such a display is only dealing withamplitude or intensity of images, it is a two-dimensional image withoutdepth of view. When an auto-stereoscopic method is used to create a 3Ddisplay, such as images in 3D movies based on binocularity of human eyesto create the 3D effect, it can cause fatigue on the eyes and would nothave the true three dimensional scenes with multiple focuses of naturalview.

As far as images displayed in 3D are concerned, there are significantlydifferences between “natural viewing or holographic 3D-display” and“autostereoscopic viewing with 3-D stereo display” due to depth cuemismatch of convergence and accommodation, resulting in the eye strainand fatigue. FIG. 1A provides an illustration 100 between naturalviewing 102 and autostereoscopic viewing 104 with 3-D stereo display. Itis well known that images formed on the retinas of two human eyes are 2Dimages. While the two eyes are seeing slightly different images, thefusion of these two images via the brain leads to depth perception,which is commonly referred to as binocular parallax. In other words, thebrain uses binocular disparity to extract depth information from thetwo-dimensional retinal images in stereopsis.

The motion of head leads to different perspectives, which is commonlyreferred to as motion parallax. Both depth perception and motionparallax are needed in the real 3D displays. Any unreal 3D display cancause conflict in brain, leading to discomfort and possibly headache(fatigue). The current autostereoscopic viewing is missing depthperception and motion parallax.

In human vision, there are another two factors that determine natural 3Dviewing: convergence and accommodation. In ophthalmology, convergence isthe simultaneous inward movement of both eyes toward each other, usuallyin an effort to maintain single binocular vision when viewing an object.Accommodation is the process by which a human eye changes optical powerto maintain a clear image or focus on an object as its distance varies.FIG. 1B summarizes these four factors that control the human naturalviewing or a real 3D viewing.

Autostereoscopy is any method of displaying stereoscopic images (addingbinocular perception of 3D depth) without the use of special headgear orglasses on the part of the viewer. As one can see from FIG. 1B,autostereoscopy is not real 3D display. Besides missing depthperception, there is one viewing perspective (no motion parallax) andthe 3D depth perception remains fixed as the viewer moves around.Further, there are convergence/focus Issues. When a viewer focuses on ascreen, his/her convergence on the object appears away from screen,resulting in conflicting information in the brain, hence often headacheand/or motion sickness.

Multi-windows, image integral types, or holographic displays with imagesof continuous depth of focus or multiple focuses can achieve real 3Ddisplay like natural viewing. Such multi-focus depth images displayoften requires processing images with phase modulation to generate thereal 3D display. However, such a technology is still at experimentallevel and not matured enough for real applications. Also, it is wellknown that the phase only modulation is in general not generating goodimage quality, often leading to mosaic-like images in low resolution.FIG. 1C shows a classic image 110 and an exemplary result 112 ofapplying phase modulation (PM) to the original image 110. Evidently, anexemplary result 114 looks much better when both of the amplitudemodulation (AM) and phase modulation (PM) are applied to the originalimage 110.

In order to combine both AM and PM, there are many researches andexperiments by utilizing two SLM devices, one to perform the AM, and theother to perform the PM only to establish an optical complicated systemtrying to achieve the good result. Although it is somehow successful incertain relative degree with a tedious and complicated set up in anoptical lab to generate either reproduced holographic display orcomputer generated holography (CGH) display but still it is unable toovercome the problem of image superimposed issues and poor image qualitydue to two separate SLM devices located apart, resulting in differencesin light traveling path, pixel to pixel alignment, wavelength deviation,and color aberration, besides a complicated and precision set-up ofoptical systems, such as 4f SLM optical systems or simply 4f systems inthe industry.

FIG. 1D shows two exemplary systems using optical interconnection tocombine both amplitude-only and phase-only SLMs. In a perspective, twoSLMs are optically connected in series by an optical relay system suchas the conventional 4-F system. Evidently, a system employing such anarchitecture would be bulky and high in cost. Further such a serialcombination of two independent modulations results in still an image ofpoor quality. FIG. 1E shows a comparison between an image with phasemodulation only and an image with such a serial combination of twoindependent phase and amplitude modulations. What is needed is anarchitecture modulating both amplitude and phase in spatial lightmodulation without using such two independent modulations coupled inseries.

As a result, there is a strong need for a technique that can performboth AM and PM in a single SLM device or panel. Such a technique isstrongly welcomed in many applications, especially in 3D holographicswith clear, compact volume, high light efficiency, high contrast andhigh resolution real 3D holographic display.

In holographic displays, the extent of angular view-ability remains oneof the most important factors affecting the experience of observingoptically reconstructed holographic images. Due to the rapid progress inliquid crystal display technologies, spatial light modulators (SLM),such as LCoS devices, provide high accuracy in optical wavereproduction. However, the resolution of the display is often limited bya number of pixels and the finite pixel pitch of a particular SLM deviceused in the display implementation. Thus there is a further need fortechniques that can display 3D holographics in higher resolution thanthe original resolution of an SLM device.

SUMMARY OF THE INVENTION

This section is for the purpose of summarizing some aspects of thepresent invention and to briefly introduce some preferred embodiments.Simplifications or omissions in this section as well as in the abstractand the title may be made to avoid obscuring the purpose of thissection, the abstract and the title. Such simplifications or omissionsare not intended to limit the scope of the present invention.

The present invention is generally related to architecture and designsof modulating both amplitude and phase in spatial light modulationwithout using two independent modulations coupled in series. Accordingto one aspect of the present invention, light propagation is controlledin two different directions (e.g., 45 and 0 degrees) to perform bothamplitude modulation (AM) and phase modulation (PM) at the same time inliquid crystals. According to another aspect of the present invention, amask is used to form an array of embossed microstructures or a pattern,where the pattern includes an array of alignment cells, a first group ofthe cells are aligned in the first direction and a second group of thecells are aligned in the second direction. Depending on applications,two cells from the first group and the second group may correspond to asingle pixel or two neighboring pixels, resulting in amplitudemodulation and phase modulation within the pixel or within an array ofpixels.

According to still another aspect of the present invention, the patternin the mask includes a plurality of two differently aligned cells, onefor the AM and the other for the PM. Each of the cells corresponding toa single pixel, causing light to be modulated in amplitude and phase atthe same time within an array of pixels. Depending on implementation,the two differently aligned cells may be arranged in different waysincluding alternating across the entire array, between columns or rowsof pixels, or according to a predefined pattern.

According to still another aspect of the present invention, the patternin the mask includes groups of four pairs, each pair having twodifferently aligned cells, corresponding to a single pixel, causinglight to be modulated in amplitude and phase, respectively in quadruple,at the same time within the pixel.

According to still another aspect of the present invention, the mask iscreated by using a similar process in semiconductor includingdeposition, etching and curing to create a designated pattern, whereembossed microstructures or specific alignment cells are created anddeposed on top of an alignment layer.

According to still another aspect of the present invention, the embossedmicrostructures or specific alignment cells are created with an e-beamtechnique to create very small structures in a resist that can besubsequently used or transferred to a substrate material to facilitatethe modulations of both AM and PM in the liquid crystals.

According to still another aspect of the present invention, instead ofusing a photo mask, a nano-stamping technique, also referred to hereinas nano-imprinting lithograph (NIL), is used to create two gratings tocause light modulations in amplitude and phase. Such two gratings may beapplied to a single pixel or every two pixels across an entire array.

According to still another aspect of the present invention, instead ofusing a photo mask or nano-stamping technique, the voltage applied orcoupled across liquid crystals is so controlled that the liquid crystalsare caused to perform the AM in a range and the PM in another range whenthe voltage is gradually increased, where the characteristics of theliquid crystals is significant, for example, by increasing the thicknessor gap of a layer of liquid crystals.

According to still another aspect of the present invention, an inputimage is first expanded into two frames based on the architecture ofsub-pixels. A first frame is derived from the input image while thesecond frame is generated based on the first frame. These two frames areof equal size to the input image and displayed as AM image and PM imagealternatively at twice the refresh rate originally set for the inputimage.

According to yet another aspect of the present invention, drivingcircuits in analog and digital are described, where each of the drivingcircuits may be used to apply a controlled voltage across one or moreliquid crystals (as a pixel) to facilitate the AM and/or PM modulationsin the liquid crystals.

The present invention may be implemented as an apparatus, a method, anda part of system. Different implementations may yield differentbenefits, objects and advantages. According to one embodiment, thepresent invention is a method in a spatial light modulator (SLM), themethod comprises designating a first group of pixels performingamplitude modulation (AM); and designating a second group of pixelsperforming phase modulation (PM), wherein both of the AM and the PM takeplace substantially at the same time, the first group of pixels and thesecond group of pixels are within a single array, both of the AM and thePM are performed via a layer of liquid crystals sandwiched between atransparent electrode layer and a electrode layer, where the electrodecomprises an array of pixel electrodes, each controlling one of thepixels, and is built on a silicon substrate.

According to another embodiment, the present invention is a spatiallight modulator (SLM) comprising a first group of pixels performingamplitude modulation (AM); and a second group of pixels performing phasemodulation (PM), wherein both of the AM and the PM take placesimultaneously, wherein the first group of pixels and the second groupof pixels are within a single array, both of the AM and the PM areperformed via a layer of liquid crystals sandwiched between atransparent electrode layer and a reflecting electrode layer, where thereflecting electrode comprises an array of pixel electrodes, eachcontrolling one of the pixels, and is built on a silicon substrate.

According to still another embodiment, the present invention is aspatial light modulator comprising a layer of liquid crystals of apredefine thickness sandwiched between a transparent electrode layer anda reflecting electrode layer, an alignment layer deposed on top of thelayer of liquid crystals, and a photo mask including a plurality ofalignment cells in rows and columns and deposed on the alignment layer,wherein the reflecting electrode comprises an array of pixel electrodes,each controlling one of pixels in two-dimensional (2D) array, and isbuilt on a silicon substrate, a first group of the alignment cells areoriented in a first direction and a second group of the alignment cellsare oriented in a second direction, light going through the first groupof the alignment cells is modulated in amplitude thereof and the lightgoing through the second group of the alignment cells is modulated inphase thereof, all via the liquid crystals and at the same time.

According to still another embodiment, the present invention is a methodin a spatial light modulator, the method comprises deposing a mask ontop of an alignment layer, wherein the alignment layer is provided to alayer of liquid crystals, with a predefined thickness, sandwichedbetween a transparent electrode layer and a reflecting electrode layer,the reflecting electrode comprises an array of pixel electrodes, eachcontrolling one of pixels in two-dimensional (2D) array, and is built ona silicon substrate, and wherein the mask includes a plurality ofalignment cells in rows and columns, a first group of the alignmentcells are oriented in a first direction and a second group of thealignment cells are oriented in a second direction, light going throughthe first group of the alignment cells is modulated in amplitude thereofand the light going through the second group of the alignment cells ismodulated in phase thereof, all via the liquid crystals and at the sametime.

According to still another embodiment, the present invention is aspatial light modulator comprising: a layer of liquid crystals, with apredefined thickness, sandwiched between a transparent electrode layerand a reflecting electrode layer, wherein the reflecting electrodecomprises an array of pixel electrodes, each controlling one of pixelsin two-dimensional (2D) array, and is built on a silicon substrate; andat least a first layer of grating attached to the layer of liquidcrystals, wherein the first layer of grating is formed by a layer oftransparent material imprinted with a stamp, wherein the first layer ofgrating includes a shallow or short-pitch grating for liquid crystalalignment superimposed onto a deeper or large-pitch grating so as tocause light modulations in amplitude and phase.

According to still another embodiment, the present invention is a methodfor making a spatial light modulator, the method comprising: providing alayer of monomer liquid on a transparent substrate; pressing a stamperonto the monomer liquid to create imprints in the monomer liquid; curingthe monomer liquid to create the first layer of grating with ashort-pitch grating and a large-pitch grating; and forming on top of thefirst layer of grating a layer of liquid crystals, with a predefinedthickness, wherein light is modulated in amplitude and phase with theshort-pitch grating and the large-pitch grating via the liquid crystals.

According to still another embodiment, the present invention is a methodfor modulating light in amplitude and phase simultaneously in liquidcrystals, the method comprises: coupling a voltage across a layer ofliquid crystals, where the voltage is supplied from a driving circuit,the voltage includes a plurality of step volts from a voltage level VLto VH and is impulsively increased to VP, VL is at least above Vth, athreshold voltage, VH is a voltage level that causes to twist the liquidcrystals, and VP is a voltage level greater than the voltage level VHfor a moment to minimize or eliminate a free relaxation time that wouldotherwise happen when the voltage level VH drops. The method furthercomprises increasing the voltage level to twist the liquid crystalsgradually from substantially blocking the incident light to fullypassing incident light therethrough and again from substantiallyblocking the incident light to fully passing the incident lighttherethrough.

According to yet another embodiment, the present invention is a spatiallight modulator for modulating light in amplitude and phasesimultaneously in liquid crystals, the spatial light modulatorcomprises: a layer of liquid crystals; a power source generating avoltage coupled across the layer of liquid crystals, where the voltageincludes a plurality of step volts from a voltage level VL to VH and isimpulsively increased to VP, VL is at least above Vth, a thresholdvoltage, VH is a voltage level causing to twist the liquid crystals, andVP is a voltage level greater than the voltage level VH for a moment tominimize or eliminate a free relaxation time that would otherwise happenwhen the voltage level VH drops; and a controller to control the powersource to increase the voltage level VP to twist the liquid crystalsgradually from substantially blocking an incident light to fully passingthe incident light therethrough and again from substantially blockingthe incident light to fully passing the incident light therethrough.

There are many other objects, together with the foregoing attained inthe exercise of the invention in the following description and resultingin the embodiment illustrated in the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood with regard to the followingdescription, appended claims, and accompanying drawings where:

FIG. 1A provides an illustration between natural viewing andautostereoscopic viewing with 3-D stereo display;

FIG. 1B summarizes four factors that control the human natural viewingor a real 3D viewing;

FIG. 1C shows an exemplary result of applying phase modulation to aclassic original image;

FIG. 1D shows an exemplary prior-art architecture using opticalinterconnection to combine both amplitude and phase SLMs, where two SLMsare optically connected in series by an optical relay system, such asthe conventional 4-F system;

FIG. 1E shows a comparison between an image with phase modulation onlyand an image with such a serial combination of two independent phase andamplitude modulations;

FIG. 2A shows an exemplary LCoS structure producing 2-dimensionaloptical image (i.e., 2D varying intensities of light or modulated lightto get gray levels of images);

FIG. 2B.1 shows an exemplary cross view of an LC layer with thealignment layer, where the pretilt alignment (rubbing) angle dictatesthe characteristics of light going through the LC molecules;

FIG. 2B.2 shows functional layers in an exemplary LCoS;

FIG. 2C, it shows an example of how an LCoS may be modified orredesigned to implement one embodiment of the present invention;

FIG. 2D shows an exemplary 8×8 array of alignment cells (eachcorresponding to one pixel;

FIG. 2E shows an array of alignment cells, each of the cells designedfor both AM and PM, which in operation appears to have dissected eachpixel into two parts alternating across an entire SLM device, allowingone half of the pixel to perform the AM and the other half to perform PMat the same time, with different ratios of AM to PM;

FIG. 2F shows two separate illustration curves, one being a reflectancecurve as AM and the other being the phase curve as PM;

FIG. 2G shows exemplary alignment cells for the pixels of FIG. 2F, eachor all can be turned, resulting in different reflectance (ortransmittance in LCoS case) and phase curves as AM and PM that happen atthe same time;

FIG. 2H shows a simulation result on a single pixel without involvingthe neighboring pixels;

FIG. 2I shows an exemplary implementation of using a photo alignmentmask method;

FIG. 2J shows a flowchart or process of creating an SLM deviceperforming both AM and PM simultaneously within a cell or within anarray according to one embodiment;

FIG. 3A shows how the LCs corresponding to a pixel perform quadruple AMand PM at the same time and increases the image resolution forholographic display in better quality;

FIG. 3B shows that a pixel size has to be around 1 μm for a reasonableholographic display in 4K×2K or even as high as 421K×316K;

FIG. 3C shows different FOVs corresponding to different pitch sizes,given A=450 nm;

FIG. 4A illustrates how nano-imprinting lithograph (NIL) is used toimplement one embodiment as a process to alter the characteristics ofthe liquid crystals used to module light in AM and PM;

FIG. 4B shows an exemplary tamp designed to form two different images,one for AM and the other for PM;

FIG. 4C shows a cross-section view of two pixels, each implemented withnano-imprinting lithograph (NIL);

FIG. 4D shows an enlarged view of one pixel in which the gratings may beimprinted on both sides of the LC layer;

FIG. 5 summarizes some of the exemplary implementations;

FIG. 6A shows an Electro-Optical (EO) curve based on a layer of liquidcrystals commonly used in a prior art SLM device;

FIG. 6B shows an EO curve and a phase curve based on Equation (1) and anEO curve based on Equation (2);

FIG. 6C shows two examples of applied voltages, where one voltageapplied causes a free relaxation time after the voltage is released, andthe other voltage has an impulsive higher volt near its end;

FIGS. 7A and 7B show respectively two functional diagrams for the analogdriving method and digital driving method for SLM devices (e.g., LCoSpanels);

FIG. 8A shows a functional block diagram of an image element accordingto one embodiment of the present invention;

FIG. 8B and FIG. 8C are used to show how an input image is expanded toachieve the perceived resolution;

FIG. 8D illustrates an image of two pixels, one being full intensity(shown as black) and the other one being one half of the full intensity(shown as grey);

FIG. 8E, it shows another embodiment to expand an input image bycontrolling the intensities of the expanded two images;

FIG. 8F shows a flowchart or process of generating two frames of imagefor display in an improved perceived resolution of an input image;

FIG. 9A shows an exemplary control circuit to address the sub-pixelelements;

FIG. 9B shows some exemplary directions a pixel (including a group ofsub-pixels) may be shifted by a sub-pixel in association with togglingcontrol signals WL_SWITCH and BL_SWITCH;

FIG. 10A shows a circuit implementing the pixels or pixel elements withanalog sub-pixels;

FIG. 10B shows two pixel elements A and B each including foursub-pixels, where one sub-pixel is the pivoting pixel 1014 shared ineach of the two pixel elements A and B;

FIG. 10C shows a simplified circuit from the circuit of FIG. 10Aaccording to one embodiment of the present invention;

FIG. 11A shows a circuit implementing the pixels or pixel elements withdigital sub-pixels, each of the sub-pixels is based on a digital memorycell (e.g., SRAM);

FIG. 11B shows a concept of sharing the pivoting sub-pixel in two pixelelements; and

FIG. 11C shows an exemplary circuit simplified from, the circuit of FIG.19A based on the concept of pivoting pixel.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The detailed description of the invention is presented largely in termsof procedures, steps, logic blocks, processing, and other symbolicrepresentations that directly or indirectly resemble the operations ofoptical signals, light, signals or data processing devices. Theseprocess descriptions and representations are typically used by thoseskilled in the art to most effectively convey the substance of theirwork to others skilled in the art.

Reference herein to “one embodiment” or “an embodiment” means that aparticular feature, structure, or characteristic described in connectionwith the embodiment can be included in at least one embodiment of theinvention. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment, nor are separate or alternative embodiments mutuallyexclusive of other embodiments. Further, the order of blocks in processflowcharts or diagrams representing one or more embodiments of theinvention do not inherently indicate any particular order nor imply anylimitations in the invention.

The present invention is generally related to devices that can beadvantageously used in holographic displays of high quality images basedon two simultaneous spatial light modulations (SLM), where themodulations include both amplitude modulation (AM) and phase modulation(PM) that occur simultaneously in a single device or panel (e.g., amicrodisplay) or even within each of the individual pixels in a singleSLM microdisplay panel. Various implementations are described.Supporting circuitry is also described herein to support suchsimultaneous AM and PM.

The present invention may be implemented in a single SLM device or panel(microdisplay). To facilitate the description of the present invention,an example of such SLM microdisplay, LCoS, is used herein forillustration. Those skilled in the art can appreciate that thedescription herein is equally applicable for other types of displaydevices demonstrating similar characteristics as in SLM device or LCoS.

LCoS or LCOS stands for liquid crystal on silicon, a miniaturizedactive-matrix liquid-crystal display or “microdisplay” using a liquidcrystal layer on top of a silicon backplane. It is also referred to as aspatial light modulator. In an LCoS display, a semiconductor chip (e.g.,CMOS) controls the voltage on square reflective aluminum electrodesburied just below the chip surface, each controlling one pixel. Forexample, a chip with XGA resolution will have 1024×768 plates, each withan independently addressable voltage. Typical cells are about 1-3centimeters square and about 2 mm thick, with pixel pitch as small as2.79 μm or smaller, and a (LC) layer with cell (gap) thickness being 1˜3μm. A common voltage for all the pixels is supplied by a transparentconductive layer made of, for example, indium tin oxide (ITO) on a coverglass.

LCoS can be a transmissive or reflective device, generating modulatedlight from a light source (e.g., white, RGB lights or lasers) for a lensor prism to collect, where the light is then displayed on a medium as animage. To facilitate the description of the present invention, the LCoSis deemed as a reflective device, namely an incident light is impingedupon thereon and reflected by the LCoS to form an optical image (a.k.a.,modulated light). One of the important advantages, benefits andobjectives in the present invention is that the modulated light is fromtwo simultaneous modulations in amplitude and the phase.

FIG. 2A shows an exemplary structure 200 of LCoS. In a perspective, theLCoS produces 2-dimensional optical image (i.e., 2D varying intensitiesof light or modulated light). It is very well known, a digital image canbe transmitted via a data cable but an optical image cannot betransmitted via such a data cable. In general, an optical image can betransported via an optical medium (e.g., air, waveguide or fibers)depending on applications. Instead of using tiny mirrors that turn onand off, LCoS uses liquid crystals as light modulators by turning anglesthereof to control the amount of reflected light.

A liquid crystal (LC) is a substance that is in mesomorphic state (notexactly a liquid or a solid). Its molecules usually hold their shape,like a solid, but they can also move around, like a liquid. Nematicliquid crystals, for example, arrange themselves in loose parallellines. A layer of liquid crystals (or a LC layer) is positioned,sandwiched or coupled between a transparent electrode layer and areflecting electrode layer, where the reflecting electrode comprises anarray of pixel electrodes and is built on a silicon substrate. It shouldbe noted that there are other layers integrated with the LC layerbetween the transparent electrode layer and the reflecting electrodelayer. As used herein, the term “positioned”, “sandwiched” or “coupled”between two layers does not mean there is only one item between the twolayers. Other layers of materials or components may be added on top ofthe item or sandwich the item to alter, modify or enhance the behavior,performance or characteristics of the item, all between the two layers.When placed between two polarized layers, the twisted crystals guide thepath of light. When a voltage difference is applied between thetransparent electrode layer and one pixel electrode, LC moleculestherebetween are re-orientated with an applied electric field. Bychanging the direction of the light, the crystals allow or prevent itspassage therethrough.

The molecules of liquid crystals are usually much longer than they arewide. In a rod-like liquid crystal, the molecules are oriented in thesame direction locally, giving rise to optical birefringence, i.e. theindex of refraction along the long axis of the molecule is significantlydifferent than the optical index perpendicular to it. In another words,birefringence is the optical property of a material having a refractiveindex that depends on the polarization and propagation direction oflight. Without further getting into the details of the molecules and/orthe liquid crystals and how they affect the birefringence, which beyondthe scope of the present invention, it is known that controlling how thelight or the polarization and propagation direction of light enteringthe liquid crystals dictates the reflectance or transmittance of thelight going through the LC layer.

When a voltage difference is applied between the transparent electrodelayer and one pixel electrode, LC molecules therebetween arere-orientated with an applied electric field. By changing the directionof the light, the crystals allow or prevent its passage therethrough.Since the LC is birefringent, the orientation results in a phase shift,commonly known phase retardation, to the light, where the phaseretardation is controllable by the voltage difference due to theElectric Controlled Birefringence Effect, ECB Mode).

When a linear polarized incident light enters the LC layer at an angleof ϕ to the director axis of the liquid crystal, it is split into twobeams with different polarizations, namely the extraordinary wave(E-light), in which the polarization direction is parallel to the liquidcrystal axis, and the ordinary wave (O-light), in which the polarizationdirection is perpendicular to the axis. Since the E-light and theO-light pass through the liquid crystal with different velocities, theirindices of refraction are different. Consequently, a phase difference δexists between the two waves when they emerge from the liquid crystal,i.e.:

$\begin{matrix}{\delta = {{2\;\pi\;{d( {\frac{1}{\lambda_{e}} - \frac{1}{\lambda_{o}}} )}} = {{2\pi\; d\frac{( {n_{e} - n_{o}} )}{\lambda_{v}}} = \frac{2\pi\; d\;\Delta\; n}{\lambda_{v}}}}} & {{Eq}.\mspace{14mu}(1)}\end{matrix}$where d is the cell gap (i.e., the thickness of the LC layer), Δndepends on the applied voltage, the temperature, and the wavelength ofthe incident light λ_(v), and is given by Δn=n_(e)−n_(o), which is alsoreferred to as birefringence.

When a homogeneous cell is sandwiched between two polarizers, thenormalized light transmittance is governed by the following equation:T=cos² X−sin 2β sin 2(β−X)sin²(δ/2);  Eq. (2)where X is the angle between the polarizer and an analyzer, β is theangle between the polarizer and the LC directors, and δ is the phaseretardation expressed in the Equation 1. For the simplest case that β=45degrees and the two polarizers are either parallel (X=0) or crossed(X=90), the normalized light transmittances are simplified to:T _(//)=cos²(δ/2);  Eq. (3)T _(⊥)=sin²(δ/2);  Eq. (4)

As further shown in FIG. 2A, there is an essential part, the alignmentlayer, that dictates the macroscopic uniform alignment of liquidcrystalline molecules (mesogens) near its surface, essentially toorientate the LC molecules with a specific pretilt angle, which is theangle between the director of the LC molecules and the alignment layers.FIG. 2B.1 shows an exemplary cross view of an LC layer with thealignment layer, where the pretilt alignment dictates thecharacteristics of light going through the LC molecules. Differentpretilt alignment angles may produce very different modulated lights, sodoes the thickness of the LC (e.g., a corresponding light paththerethrough). There are ways of forming the surface alignment layer.One example is to use unidirectional mechanical rubbing of thinpolyimide coating. A thin film is spin coated and then cured atappropriate temperature according to the polyimide type. Thereafter, thecured film is rubbed with a velvet cloth, producing micro or nanogrooves along the rubbing direction to which the LC molecules arealigned accordingly. FIG. 2B.2 shows functional layers in an exemplaryLCoS.

Referring now to FIG. 2C, it shows an example of how an LCoS 200 can bemodified or redesigned to implement one embodiment of the presentinvention. As in a prior art LCoS, an alignment layer 202 is deposed ontop of a layer of liquid crystals 203 (i.e., a LC layer) to configurethe liquid crystals to have pre-determined pretilt alignment angles overthe entire array of pixels. An incident light is transmitted through theLC layer with almost zero absorption. The integration ofhigh-performance driving circuitry allows the applied voltage to bechanged on each pixel, thereby controlling the phase retardation of theincident wavefront across the device. Currently, there are two types oflight modulation using LCoS devices, amplitude modulation (AM) and phasemodulation (PM). In the AM case, the amplitude of the light signal ismodulated by varying the linear polarization direction of the incidentlight. In the PM case, the phase delay is accomplished by electricallyadjusting the optical refractive index along the light path. The detailsof how an incoming light is modulated by the liquid crystals (LC) or inthe LC layer are not to be further described to avoid obscuring aspectsof the present invention. One of the objectives, benefits and advantagesin the present invention is to control the pretilt alignment angles viaa modified alignment layer or an array of alignment cells integratedwith an alignment layer. To facilitate the description of the presentinvention, for AM, all the alignment cells are diagonally aligned (e.g.,neither horizontally nor vertically aligned, or between 20-60 degrees),and for PM, all the alignment cells are horizontally aligned, meaningstarting from 0 degrees to 360 degrees and beyond.

It shall be noted, throughput the description herein, that an alignmentlayer is used as a substrate to form or hold the alignment cells (orembossed microstructures). Those skilled in the art can appreciate fromthe description herein that the alignment cells being described may verywell be incorporated into the alignment layer when it is designed orformed. To facilitate the description of the present invention, thealignment cells are assumed to be formed on top of an alignment layer.

According to one embodiment of the present invention, the twodifferently aligned cells are arranged in such a way 206 that theiralignments alternate across the entire alignment layer, namely thealignment of each cell is different from that of its neighboring cells.In other words, the alignments of the cells are alternating from AM toPM. In operation, both AM and PM happen simultaneously when a light goesthrough these cells and the LC layer is applied with proper voltages orcurrents. One of the advantages, benefits and objectives in the presentinvention is to have both AM and PM happen at the same time in an SLMdevice (e.g., LCoS panel). As all of the light is simultaneouslymodulated in phase and amplitude, a holographic image reproduced fromsuch an implementation can be in high resolution with high efficiency.

FIG. 2D shows an exemplary 8×8 array of alignment cells, eachcorresponding to one pixel. According to one embodiment shown in FIG.2D, the alignment cells for AM and PM are alternating across an entireSLM device, namely alternating pixels in odd and even row or columnwithin one SLM device. In a perspective, one half of the pixels performthe AM and the other half of the pixels perform the PM at the same time.In some modified embodiments, the alignment cells may be randomly chosenfor AM and PM or a desire pattern is designed to define certain pixelsor groups of pixels for AM or PM.

According to another embodiment shown in FIG. 2E, the alignment cellsfor AM and PM are within one single pixel by dissecting each pixel, forexample, into two parts alternating across an entire SLM device,allowing one half of the pixel to perform AM and the other half toperform PM at the same time. Depending on the implementation, thepercentage of one pixel for performing AM or PM may be 50% or predefinedper a desired performance, some of which are also shown in FIG. 2E.

As far as light efficiency is concerned, it is estimated according toone prior art system that the holographic display based on amplitudemodulation is very low (e.g., roughly only 5%) while the efficiency isincreased (e.g., to 95%) based on phase modulation. With the integratedAM and PM, the light efficiency can be considerably increased withoutlosing the resolutions.

FIG. 2F shows two separate illustration curves 242 and 244, one being areflectance curve 242 and the other being the phase curve. Under anappropriate electrode voltage (e.g., 0˜2.5V, 0˜5V or etc.), one half ofa pixel 246 performs the AM by turning the corresponding LCs from blackgradually to white or white gradually to black while the other half ofthe pixel 246 performs the PM by turning the corresponding LCs from 0gradually to 2π or 2π gradually to 0. As shown in FIG. 2E, the ratio ofAM and PM on a single pixel is not fixed at 50:50, but may be anynumbers (e.g., 40:60 or 70:30). FIG. 2G shows the alignment cells forthe pixel 246 of FIG. 2F is turned, resulting in different reflectanceand phase curves, to achieve a different desired result.

FIG. 2H shows a simulation result on the single pixel 246 withoutinvolving the neighboring pixels. The simulation shows the liquidcrystals 250 corresponding to the PM (left portion) are orienteddifferently from the liquid crystals corresponding to the AM (rightportion) when the applied voltage is changing from 0V to 5V. It shouldbe noted that the thickness or depth of the LC layer 250 is preferablytwice as much as otherwise it is used for single modulation in onepixel. According to one embodiment, it is assumed that the depth of theLC layer 250 is great 2D, where D is the depth of an LC layer used foronly AM or PM via one pixel or an array of pixels. In other words,slightly larger than this twice thickness to make sure a phase shift(0˜2π) is achieved). The corresponding reflectance curve 252 and phasecurve 254 are also presented with two voltages V1=0 and V2=5. Thephysical size of the pixel 246 is assumed that the pixel is 6.4 μm inwidth, thus approximately one half of 6.4 μm in width is dedicated forAM and the second half of 6.4 μm in width is dedicated for PM, if theratio is kept to be 50:50.

FIG. 2I shows an exemplary implementation 260 of using a photo alignmentmask method. An alignment layer 262 is added thereon with a photo mask264. Given a predefined pattern imprinted upon the photo mask 264, forexample 50/50 for each cell, namely a cell 262 is configured to causeboth of the AM and PM to take place simultaneously. The photo mask 264is etched with UV lights or other means 268. As a result, a pixel iscovered with an alignment cell 270 that has two different alignments,one 272 for the PM and the other 274 for the AM. The same pattern isetched for all the cells across the alignment layer in a single SLMdevice, as shown similarly in FIG. 2E. In an alternative embodiment (notshown in FIG. 2I), such an alignment cell, either horizontally alignedor diagonally aligned, covers just one pixel. All neighboring cells,each covering a single pixel, can be aligned differently. In otherwords, all the alignment cells are alternatively aligned eitherhorizontally for the PM or diagonally for the AM, as shown similarly inFIG. 2D.

FIG. 2J shows a flowchart or process 280 of creating an SLM deviceperforming both AM and PM simultaneously within a cell or within anarray according to one embodiment. The process 280 may be betterunderstood in conjunction with the figures above. The process 280 startswhen an implementation of adding a photo mask on top of an alignmentlayer.

An SLM device, such as LCoS, includes an LC layer to control the pass ofthe reflected (or transmitted) light. As described above, one embodimentis to modify or add an alignment layer on top of the LC layer. Dependingon the resolution of the SLM, there are a plurality of alignment cells,each responsible for a pixel. These cells need to be controlled uniquelyto facilitate the LCs in the LC layer to modulate the reflected light inamplitude and phase, given the characteristics of the underlying LCs.

At 282, a photo mask is laid over the alignment layer. As describedabove with respect to FIGS. 2D and 2E, there are two ways to have the AMand PM occur at the same in a single SLM device, one being within a celland the other within an array of cells. To facilitate the description ofthese two embodiments, the term “cell-based simultaneous modulations”indicates that the AM and PM are performed within a cell at the sametime, namely an alignment cell is partitioned to have two parts, one forthe AM and the other for the PM and the same is repeated across theentire array of cells in a single SLM device. The term “array-basedsimultaneous modulations” indicates that all the cells are alternativelydesignated to perform the AM or PM, namely no neighboring alignmentcells performing the same modulations.

At 284, the process 280 determines how to design or configure the photomask via print or lithography. If it is decided to have the cell-basedsimultaneous modulations, the process 280 goes to 286, where acorresponding pattern can be printed on the photo mask. According to oneembodiment, all the cells shall have the same pattern across the array.According to another embodiment, all the cells in a row have the samepattern across the row while the neighboring rows have a half-pixelshift pattern in view of FIG. 2E, resulting in two alternating patternsacross the rows. If it is decided to have the array-based simultaneousmodulations, the process 280 goes to 288, where a corresponding patterncan be printed on the photo mask. The pattern dictates that every othercell is designated to perform one modulation (e.g., the AM) and everyanother cell is designated to perform another modulation (e.g., the PM).FIG. 2D shows an exemplary portion of such a pattern.

The pattern may be varying depending on what performance is beingdesired. In general, the ratio of the AM and PM within one cell is50/50, but the ratio of the AM and the PM may be adjusted to any numberif desired. Once the pattern is decided, the pattern may be imprintedonto the photo mask. The detail of making or imprinting a pattern onto aphoto mask is not to be further described herein as it is well known inthe art (e.g., in semiconductor manufacturing). The process 280 now goesto 290, where the photo mask is etched. There are many ways to etch aphoto mask. Again, the detail of etching a photo mask is not to befurther described herein as it is well known in the art (e.g., insemiconductor manufacturing). As a result of the alignment layer withthe designated aligned cells, an SLM device performing the AM and the PMsimultaneously within a cell is created at 292 or an SLM deviceperforming the AM and the PM simultaneously within an array is createdat 294.

Referring now to FIG. 3A, it shows another embodiment 300 of thealignment cells, each of the cells is divided into four equal parts,each having its own alignment cell, where each cell causes both AM andPM to happen at the same time, resembling the concept of subpixels. As aresult, a pixel could be perceived as if it is divided into foursubpixels. If the width of the pixel is 4 μm, each of the subpixelswould be approximately around 1 μm. As shown in FIG. 3B, the pixel sizehas to be around 1 μm for a reasonable holographic display in 4K×2K.With the original pixel size, it would be difficult to achieve such areasonable resolution for a holographic display. With the presentembodiment, as will be further described below, it is now possible toachieve the resolution with a specially designated mask. Those skilledin the art can appreciate that a mask can be readily modified to makethese alignment cells with quadruple subcells given the descriptionabove.

One of the advantages, benefits and objectives using a smaller pixelsize by the subcell concept via the photo mask is to provide a largerfield of view (FOV). In a perspective, Bragg's law may be used toexplain the result. It is well known that the diffraction formula is sinϕ=nλ/2p, where n is a positive integer and λ is the wavelength of theincident wave, p is a pitch size. ϕ would be increased when the pitchsize p is reduced. Since FOV is approximated by 2ϕ_(max), a smallerpixel size will increase the FOV. FIG. 3C shows different FOVscorresponding to different pitch sizes, given λ=450 nm.

According to one embodiment which is also referred to as nano-imprintinglithograph (NIL), it includes two gratings: a shallow, short-pitchgrating for liquid crystal alignment superimposed onto a deeper and alarger-pitch resonant grating. Both gratings are patterned onto asilicon nitride (SiN) thin-film layer. The two gratings are in parallel.Nematic liquid crystals are introduced on top of the superimposedgratings. The two indium tin oxide (ITO) layers serve as the low opticalloss electrodes to apply an electric field across the liquid crystalcells.

According to one embodiment, the acrylic used in sculpted gratings isformed by the reaction of a monomer liquid with a polymer powder. Themonomers (“mono” meaning “one”) contained in the liquid are microscopicchemical units which react together when mixed with chemicals in thepowder. FIG. 4A illustrates how the NIL works in one embodiment as aprocess 400. A transparent substrate 402 is laid with a layer of monomerliquid 404. A stamp 406 is predesigned. In one embodiment, the stamp 406is made from polydimethylsiloxane (PDMS), a silicone. It is patternedusually against a master to form a relief pattern used in softlithography. The stamp 406 is pressed onto the monomer liquid 404. Underthe UV treatment, as shown at 408, the monomer liquid 404 is shapedaccording to the image of the stamp 406, essentially an opposite imageof the stamp 406.

The same process 400 is applied to create two alignments for the AM andPM. As shown in FIG. 4B, a stamp (such as the stamp 406) is designed toform two different images. After applied to a layer of monomer liquid,two different alignment cells 422 and 424 are shaped out of the monomerliquid. FIG. 4C shows a cross-section view of two pixels, eachimplemented with the nano-imprinting lithograph (NIL). The monomerliquid 432 is so shaped that the liquid crystals 434 are twisted ororientated differently when a varying voltage is applied across each ofthe pixels. FIG. 4D shows an enlarged view of one pixel in which the twotypes of gratings may be imprinted on the both sides 452 and 454 of theLC layer 450.

According to another embodiment, a focused beam of electrons is used todraw custom shapes on a surface covered with an electron-sensitive filmcalled a resist (exposing). The electron beam changes the solubility ofthe resist, enabling selective removal of either the exposed ornon-exposed regions of the resist by immersing it in a solvent(developing). The purpose, as with photolithography, is to createembossed microstructures (specific alignment cells) to facilitate themodulations of both AM and PM in the liquid crystals.

FIG. 5 summarizes some of the exemplary implementations that usedifferent materials. According to another embodiment that uses thecharacteristics of the liquid crystals under specific voltage control toperform the AM and PM. In this embodiment, the alignment layer 202 isnot to be modified or laid with a photo mask. As shown in 2A, the liquidcrystal layer 203 is positioned between a transparent electrode layerand a reflecting electrode layer, where the reflecting electrode layerwould be replaced with a transparent electrode in the case oftransmissive LCoS. The reflecting electrode comprises an array of pixelelectrodes and is built on a silicon substrate or TFT substrate (such asHTPS). When a voltage difference is applied between the transparentelectrode layer and one pixel electrode, LC molecules therebetween arere-orientated with an applied electric field.

FIG. 6A shows an Electro-Optical (EO) curve 602 based on a layer ofliquid crystals commonly used in a prior art SLM device. What the EOcurve 602 shows is that a voltage applied across the layer of liquidcrystal 606 goes up for white (completely turn-on) and then goes downfor black (completely turn-off), which results in gray levels of images.In other words, when the voltage 604 moves low (VL or 0V) to high (VH or2.5V), resulting in the corresponding changes of the applied electricfield across the layer of liquid crystals, the light passing the liquidcrystal 606 is increased. When the voltage 604 moves H to L, the lightpassing the liquid crystal 606 is decreased. According to one embodimentof the present invention, the voltage 604 is so controlled that itcontinues going up, causing the liquid crystals in the layer 606 toclose gradually again even when the voltage exceeds H. In a perspective,there are two driving voltages, the liquid crystals in the layer 606experience open and close, and then gradually open again as the voltage604 continues going up. Under such a controlled voltage 604, the liquidcrystals in the layer 606 provide a phase shift of at least 360 degreesor 2PI (π).

FIG. 6B shows an EO curve 612 and the phase curve 614 based on Equation(1) above and an EO curve 616 based on Equation (3) above. If δ is knownas required phase retardation said 2π, then the only parameters impactedthe whole system function are limited to three parameters Δn, d, and λ.The “d”, as cell gap of thickness of liquid crystal (LC) layer for lightsuch R, G, and B to travel or propagate through and bounce light backreturning total distance as 2d. The “Δn” as birefringence based ondiffractive index on liquid crystal characteristics and the “λ” aswavelength of light wave such as red, green and blue in differencewavelength of light selected.

As a comparison, FIG. 6C shows two examples of applied voltages 612 and614, where the voltage 612 applied causes a free relaxation time afterthe release of the voltage 612, as shown in the reflectance curve 614.According to one embodiment, the voltage source 604 is designed togenerate a voltage 614. In other words, the voltage 614 is impulsivelyincreased by a predefined level VP near its end. For example, VP isequal to 1.5 VH. The exact time may be determined based on the flashrate of a video signal. What shall be noted is the clock-type or stepvoltage 614 (between VL and VH) being applied across the liquidcrystals. In operation, the lowest voltage VL is at least above Vth, athreshold voltage. VH is the voltage that causes to twist the liquidcrystals. VP is greater than VH for a moment to minimize or eliminatethe free relaxation time.

Now referring to FIG. 7A and FIG. 7B, which are duplicated from U.S.Pat. No. 10,147,350 which is hereby incorporated by reference. FIG. 7Aand FIG. 7B show respectively two types of driving circuits to providethe voltage to drive the liquid crystals. As is the case in most memorycell architecture, image elements or pixels are best accessed viadecoding a sequence of pre-determined address bits to specify thelocation of a target image element. These pre-determined address bitsare further divided into X-address bits and Y-address bits. TheX-address bits decode the location of control line (word line) of animage element while the Y-address bits decode the location of data line(bit line) of the image element. The set of circuits that decode theX-address bits into selected control lines (word lines) is calledhorizontal decoder or X-decoder. The set of circuits that decodeY-address bits into selected data lines (bit lines) is called verticaldecoder or Y-decoder.

In general, there are two driving methods, analog and digital, toprovide a gray level to a pixel, namely an image element or pixel in anarray. As used herein, gray or a gray level implies a brightness orintensity level, not necessarily an achromatic gray level between blackand white. For example, a red color is being displayed, in which case agray level of the color means how much red (e.g., a brightness level inred) to be displayed. To facilitate the description of the presentinvention, the word gray will be used throughout the description herein.In the analog driving method, the gray level is determined by a voltagelevel stored in a storage node. In the digital driving method, the graylevel is determined by a pulse width modulation (PWM), where the mixtureof an ON state voltage duration and an OFF state voltage durationresults in a gray level through the temporal filtering of human eyes.

There are two ways to feed video signals to the pixels in a spatiallight modulator (e.g., an LCoS device): analog driving method anddigital driving method . . . . Two functional diagrams 702 and 704 forthe analog driving method and digital driving method are shown. For theanalog driving scheme, one pixel includes a pass device 706 and onecapacitor 708, with a storage node connected to a mirror circuit 710 tocontrol a corresponding liquid crystal or a set of liquid crystals in aLC layer. For the digital driving method, pulse width modulation (PWM)is used to control the gray level of an image element. A static memorycell 712 (e.g., SRAM cell) is provided to store the logic “1” or logic“0” signal periodically. The logic “1” or logic “0” signal determinesthat the associated element transmits the light fully or absorbs thelight completely, resulting in white and black. A various mixture of thelogic “1” duration and the logic “0” duration decides a perceived graylevel of the element. Via the gate 714, a mirror circuit 716 controls acorresponding liquid crystal or a set of liquid crystals in a LC layer.

As described above, FIG. 3A shows how the LCs corresponding to a pixelperform quadruple AM and PM at the same time increases the imageresolution for holographic display in better quality. Referring now toFIG. 8A that is duplicated from U.S. Pat. No. 10,147,350, it shows apixel 802 including four subpixels 804A-804D. It is now assumed that aninput image of a first resolution (e.g., 500×500) is received anddisplayed in a second resolution (e.g., 1000×1000) via the alignmentcells shown in FIG. 3A, where the second resolution is twice as much asthat the first resolution. According to one embodiment, the sub-pixelelements are shown to achieve the perceived resolution. It is importantto note that such an improved spatial resolution is perceived by humaneyes, not actually the doubled resolution of the input image. Tofacilitate the description of the present invention, FIG. 8B and FIG. 8Care used to show how an input image is expanded to achieve the perceivedresolution.

It is assumed that an input image 810 is of 500×500 in resolution.Through a data process 812 (e.g., upscaling and sharpening), the inputimage 810 is expanded to reach an image 814 in dimension of 1000×1000.FIG. 8C shows an example of an image 816 expanded to an image 818 ofdouble size in the sub-pixel elements. In operation, each of the pixelsin the image 816 is written into a group of all (four) sub-pixelelements (e.g., the exemplary sub-pixel structure of 2×2). Those skilledin the art can appreciate that the description herein is readilyapplicable to other sub-pixel structures (3×3, 4×4 or 5×5, and etc),resulting in even more perceived resolution. According to oneembodiment, a sharpening process (e.g., part of the data processing 812of FIG. 8B) is applied to the expanded image 818 to essentially processthe expanded image 818 (e.g., filtering, thinning or sharpening theedges in the images) for the purpose of generating two frames of imagesfrom the expanded image 818. In one embodiment, the value of eachsub-pixel is algorithmically recalculated to better define the edges andproduce the image 820. In another embodiment, values of neighboringpixels are referenced to sharpen an edge.

The processed image 820 is then separated into two images 822 and 824 bythe separation process 825. Both 822 and 824 have a resolution same asthat of the input image (e.g., 500×500), where the sub-pixel elements ofimages 822 and 824 are all written or stored with the same value. Theboundaries of pixel elements in the image 822 are purposely to bedifferent from the boundaries of pixel elements in the image 824. In oneembodiment, the boundaries of pixel elements are offset by half-pixel(one sub-pixel in a 2×2 sub-pixel array) vertically and by half-pixel(one sub-pixel in a 2×2 sub-pixel array) horizontally. The separationprocess 825 is done in a way that, when overlapping images 822 and 824,the combined image can best match the image 820 of quadruple resolutionof the input image 816. For the example in FIG. 8C, to keep the constantintensity of the input image 810, the separation process 825 alsoincludes a process of reducing the intensity of each of the two images822 and 824 by 50%. Operationally, the intensities in the first imageare reduced by N percent, where N is an integer and ranged from 1 to100, but practically is defined around 50. As a result, the intensitiesin the second image are reduced by (100−N) percent. These two images 822and 824 are displayed alternatively at twice the refresh rate as thatfor the original input image 810. In other words, if the input image isdisplayed at 50 Hz per second, each of pixels in two images 822 and 824are displayed 100 Hz per second. Due to the offset in pixel boundary anddata process, viewers perceive the combined image close to the image820. Offsetting the pixel boundary between images 822 and 824 has theeffect of “shifting” pixel boundary. As illustrated as two images 826and 828 according to another embodiment, the example in FIG. 8C is likeshifting a (sub)pixel in southeast direction.

Depending on implementation, the separation process 825 may be performedbased on an image algorithm or one-pixel shifting, wherein one-pixelshifting really means one sub-pixel in the sub-pixel structure as shownin FIG. 8A. There are many ways to separate an image of N×M across theintensity into two images, each of N×M, so that the perceived effect ofdisplaying the two images alternatively at the twice refresh ratereaches the visual optimum. For example, one exemplary approach is toretain/modify the original image as a first frame with reduced intensitywhile producing the second frame with the remaining from the firstframe, again with reduced intensity. Another exemplary approach is toshift one pixel (e.g., horizontally, vertically or diagonally) from thefirst frame (obtained from the original or an improved thereof) toproduce the second frame, more details will be provided in the sequel.FIG. 8C shows that two images 822 and 824 are produced from theprocessed expanded image 820 per an image algorithm while two images 826and 828 are generated by shifting the first frame on pixel diagonally toproduce the second frame. It should be noted that the separation processherein means to separate an image across its intensities to produce twoframes of equal size to the original image. FIG. 8D illustrates an imageof two pixels, one being full intensity (shown as black) and the otherone being one half of the full intensity (shown as grey). When the twopixel image is separated into two frames of equal size to the original,the first frame has two pixels, both being one half of the fullintensity (shown as grey) and the second frame also has two pixels, onebeing one half of the full intensity (shown as grey) and the other beingalmost zero percent of the full intensity (shown as white). Now thereare twice as many pixels as the original input image, displayed in acheckerboard pattern. Since each pixel is refreshed only 60 times persecond, not 120, the pixels are half as bright, but because there aretwice as many of them, the overall brightness of the image stays thesame.

Referring now to FIG. 8E, it shows another embodiment to expand an inputimage 810 by controlling the intensities of the expanded two images. Itis still assumed that the input image 810 is of 500×500 in resolution.Through the data process 812, the input image 810 is expanded to reach adimension of 1000×1000. It should be noted that 1000×1000 is not theresolution of the expanded image in this embodiment. The expanded imagehas two 500×500 decimated images 830 and 832. The expanded view 834 ofthe decimated images 830 and 832 shows that pixels in one image aredecimated to allow the pixels of another image to be generatedtherebetween. According to one embodiment of the present invention, thefirst image is from the input image while the second image is derivedfrom the first image. As shown in the expanded view 834 of FIG. 8E, anexemplary pixel 836 of the second image 832 is derived from three pixels838A, 838B and 838C. The exemplary pixel 832 is generated to fill thegap among three pixels 838A, 838B and 838C. The same approach, namelyshifting by one pixel, can be applied to generate all the pixels for thesecond image along a designated direction. At the end of the dataprocessing 812, there is an interlaced image including two images 830and 832, each is of 500×500. A separation process 825 is applied to theinterlaced image to produce or restore therefrom two images 830 and 832.

Referring now to FIG. 8F, it shows a flowchart or process 840 ofgenerating two frames of image for display in an improved perceivedresolution of an input image. The process 840 may be implemented insoftware, hardware or in combination of both, and can be betterunderstood in conjunction with the previous drawings. The process 840starts when an input image is received at 841.

The resolution of the input image is determined at 842. The resolutionmay be given, set or detected win the input image. In one case, theresolution of the input image is passed along. In another case, theresolution is given in a head file of the input image, where the headfile is read first to obtain the resolution. In still another case, theresolution is set for a display device. In any case, the resolution iscompared to a limit of a display device at 844, where the limit isdefined to be the maximum resolution the display device can displayaccording to one embodiment of the present invention.

It is assumed that the limit is greater than 2 times the resolutionobtained at 842. That means a display device with the limit can “double”the resolution of the input image. In other words, the input image canbe displayed in much improved perceived resolution than the original orobtained resolution. The process 840 moves to 846 where the pixelsvalues are written into pixel elements, where each of the pixel elementshas a group of sub-pixels. In operation, it is essentially an upscaleprocess. At 848, applicable image processing is applied to the expandedimage. Depending on implementation, exemplary image processing mayinclude sharpening, edge detection, filtering and etc. The purpose ofthe image processing at this stage is to minimize errors that may havebeen introduced in the upscale operation when separating the expandedimage into two frames. It should also be noted that the upscale processor the image processing may involve the generation of a second framebased on a first frame (the original or processed thereof) asillustrated in FIG. 8C. At the end of 848, an expanded image that hasbeen processed applicably is obtained.

At 850, the expanded image is going under image separation to form twoindependent two frames. In connection with the description above, one ofthe two images is for the AM and the other of the two images is for thePM, all via the LC layer with the photo mask. As described above, thereare ways to separate an image across the intensity into two frames ofequal size to the image. In other words if the image is of M×N, each ofthe two frames is also of M×N, where only the intensity of the image isseparated. Regardless of whatever an algorithm is used, the objective isto keep the same perceived intensity and minimize any artifacts in theperceived image when the two frames are alternatively displayed at thetwice refresh rate (e.g., from 50 frames/sec to 100 frames/sec) at 852.

Back to 844, now it is assumed the limit is less than 2 times theresolution obtained at 842. That means a display device with the limitcannot “double” the resolution of the input image. In other words, it ispractically meaningless to display an image in a resolution exceedingthat of the display device unless some portions of the image are meantto be chopped off from display. The process 840 now goes to 854 todisplay the image in native resolution. One of the objectives, benefitsand advantages in the present invention is the inherent mechanism todisplay images in their native resolutions while significantly improvingthe perceived resolution of an image when the native resolution is notof high.

It should be noted that the process 840 of FIG. 8F is based on oneembodiment. Those skilled in the art can appreciate that not every blockmust be implemented as described to achieve what is being disclosedherein. It can also be appreciated that the process 840 can practicallyreduce the requirement for the memory capacity. According to oneembodiment, instead of providing memory for storing two frames of image,only the memory for the first frame may be sufficient. The second framemay be calculated or determined in real time.

Referring now to FIG. 9A, it shows an exemplary control circuit toaddress the sub-pixel elements 900. The set of circuits that decode theX-address bits into selected control lines (word lines) is calledX-decoder 902. The set of circuits that decode Y-address bits intoselected data lines (bit lines) is called Y-decoder 904. However, one ofthe differences between the prior-art decoder circuits and FIG. 9A isthat the X-decoder 902 and Y-decoder 904 can address two lines at atime. For example, as shown in FIG. 9A, when both BL_SWITCH andWL_SWITCH are set to 0, a group of four sub-pixels 906 are selected byword line WL1 and data line BL 1. In another operation, when bothBL_SWITCH and WL_SWITCH are set to 1, a group of four sub-pixels 908 areselected.

As an example shown in FIG. 9A, each of the X-decoder 902 and Y-decoder904 address two lines simultaneously by using a multiplexor or switch905 to couple two switch signals WL1 and WL0, each of which is selectedby a control signal WL_SWITCH. Controlled by the control signalWL_SWITCH being either 1 or 0, two neighboring lines 910 or 912 aresimultaneously addressed by the X-decoder 902. The same is true for theY-decoder 904. As a result, the sub-pixel elements 906 and the sub-pixelelements 906 are respectively selected when WL_SWITCH is switched from 0to 1 and at the same time BL_SWITCH is switched from 0 to 1. In aperspective, the sub-pixel group 906 is moved diagonally (along thenortheast or NE) by one sub-pixel to the sub-pixel group 908. FIG. 9Bshows some exemplary directions a pixel (including a group ofsub-pixels) may be shifted by a sub-pixel in association with togglingcontrol signals WL_SWITCH and BL_SWITCH.

Referring back to FIG. 9A, as each time, the sub-pixel group 906 or thesub-pixel group 908 is shifted by one-half sub-pixel group or onesub-pixel, it can be observed that one sub-pixel is fixed or alwaysaddressed when WL_SWITCH is switched from 0 to 1 or 1 to 0 and BL_SWITCHis switched from 0 to 1 or 1 to 0. This fixed sub-pixel is referred toherein as a pivoting (sub)pixel, essentially one of the sub-pixels in asub-pixel group or pixel element. As will be further described below,circuitry facilitating to implement one of the embodiments in thepresent invention can be significantly simplified, resulting in lesscomponents, smaller die size and lower cost.

Referring now to FIG. 10A, it shows a circuit 1800 implementing thepixels or pixel elements with analog sub-pixels. Each of the sub-pixelsis based on an analog cell. Similar to FIG. 7A, an analog cell 1002includes a pass device 1004 and one capacitor 1006 to store a charge forthe sub-pixel. A pass device 1008 is provided to transfer the charge onthe capacitor 1006 to the mirror plate of liquid crystal 1010, which mayalso serve as a capacitor. Instead of using identical analog cells assub-pixels, the circuitry by utilizing the shared pivoting pixel of twoshift positions can be further simplified. FIG. 10B shows two pixelelements A and B each including four sub-pixels, where one sub-pixel isthe pivoting pixel 1014 shared in each of the two pixel elements A andB. It can be observed that the pivoting pixel needs to be updated byeither one of the two pixel elements A and B, and is always selected. Asa result, the circuit 1000 of FIG. 10A can be simplified to a circuit1018 of FIG. 10C according to one embodiment of the present invention.The circuit 1018 of FIG. 10C shows that three non-pivoting cells 1A, 2Aand 3A in the pixel element A are updated in accordance with the updatesignal A while three non-pivoting cells 1B, 2B, and 3B in the pixelelement B as well as the pivoting cell are updated in accordance withthe update signal B.

As further shown in FIG. 10A, there is only one capacitor 1015 to serveas the storage element and one pass gate 1016 to connect the data lineto capacitor 1015 within the two pixel elements A and B. Therefore, onlyone word line and only one data line is needed to address the storageelement 1015. Shifting is performed through switching between thecontrol signals update A and update B. When update A is 1, the videosignal stored in capacitor 1015 is passed to all sub-pixels in pixelgroup A, including sub-pixel 1A, 2A, 3A, and the pivoting (sub)pixel1014. When update B is 1, the video signal stored in capacitor 1015 ispassed to all sub-pixels in pixel group B, including sub-pixel 1B, 2B,3B, and the pivoting (sub)pixel 1014.

It can be observed that the pivoting pixel 1014 needs to be updated byeither one of the two pixel elements A and B, and is always selected. Asa result, the circuit 1000 of FIG. 10A can be simplified as only onecapacitor 1015, one pass gate 1016, one word line, and one data line areneeded to implement the sub-pixel shifting. Compared to the circuit 1000of FIG. 10A, the circuit of FIG. 10B can result in smaller area forcircuitry as less components, word lines and data lines are needed. Thecircuit 1018 of FIG. 10C shows the physical implementation of thecircuit described in FIG. 10B according to one embodiment of the presentinvention. The circuit 1018 of FIG. 10C shows that three non-pivotingcells 1A, 2A and 3A in the pixel element A are updated in accordancewith the update signal A while three non-pivoting cells 1B, 2B, and 3Bin the pixel element B as well as the pivoting cell are updated inaccordance with the update signal B. The pass gate and the capacitor areassociated to the pivoting sub-pixel for ease of illustration. Inreality, they can be placed anywhere inside the pixel group A and pixelgroup B boundary. For all non-pivoting sub-pixel cells, 1A, 2A, 3A, 1B,2B, and 3B, they are shared with neighboring pixel A and pixel B cells.Neighboring pass gates coupled with update A and update B are shown indotted lines in FIG. 10C.

FIG. 11A shows a digital version of a sub-pixel 1100. In one embodiment,pulse width modulation (PWM) is used to control the gray level of animage element. Similar to FIG. 7B, a static memory cell 1102 (e.g., SRAMcell) is provided to store a logic value “1” or “0” periodically. Thelogic value “1” or “0” signal determines that the associated element1100 transmits the light fully or absorbs the light completely,resulting in white or black. A various mixture of the logic “1” durationand the logic “0” duration decides a perceived gray level of the element1100. FIG. 11B shows the concept of using the pivoting sub-pixel. Thecircuit 1112 in FIG. 11B shows two pixel elements A and B each includingfour sub-pixels, where one sub-pixel is the pivoting pixel 1114 sharedin each of the two pixel elements A and B. It can be observed that thepivoting (sub)pixel 1114 needs to be updated by either one of the twopixel elements A and B, and is always selected. As a result, the circuit1100 of FIG. 11A can be simplified to a circuit 1112 of FIG. 11Baccording to one embodiment of the present invention. The circuit 1118of FIG. 11C is a alternative representation of the circuitry shown inFIG. 11B. The circuit 1118 of FIG. 11C shows that three non-pivotingcells 1A, 2A and 3A in the pixel element A are updated in accordancewith the update signal A while three non-pivoting cells 1B, 2B, and 3Bin the pixel element B as well as the pivoting cell are updated inaccordance with the update signal B.

The present invention has been described in sufficient detail with acertain degree of particularity. It is understood to those skilled inthe art that the present disclosure of embodiments has been made by wayof examples only and that numerous changes in the arrangement andcombination of parts may be resorted without departing from the spiritand scope of the invention as claimed. Accordingly, the scope of thepresent invention is defined by the appended claims rather than theforgoing description of embodiments.

We claim:
 1. A display device for displaying an input image in improvedperceived resolution, the display device comprising: a memory arrayhaving a plurality of pixel elements, each of the pixel elementsincluding at least 2×2 sub-pixels; an interface to the memory array, theinterface receiving a native resolution of the input image, wherein whenthe improved perceived resolution is greater than twice the nativeresolution, the interface is controlled to expand the input image intoan expanded image in the memory array by generating from the expandedimage a first frame and a second frame of image, both of the first andsecond frames being of equal size to the input image; a circuit providedto display the first and second frames alternatively at twice refreshrate originally set for the input image, wherein either one or both ofthe first frame and the second frame of images are modulated.
 2. Thedisplay device as recited in claim 1, wherein the first frame of imageis modulated in amplitude and the second frame of image is modulated inphase.
 3. The display device as recited in claim 2, wherein the firstframe of image is modulated in amplitude across all pixels in a firstspatial light modulator; and the second frame of image is modulated inphase across all of the pixels in a second spatial light modulator. 4.The display device as recited in claim 3, wherein each of the pixels inthe first frame of image or the second frame of image is configured toperform amplitude modulation or phase modulation.
 5. The display deviceas recited in claim 4, wherein the each of the pixels corresponds to twodifferently aligned cells, one for the amplitude modulation and theother for the phase modulation.
 6. The display device as recited inclaim 5, wherein one of the two differently aligned cells is diagonallyaligned for the amplitude modulation, and the other one of the twodifferently aligned cells is horizontally aligned for the phasemodulation.
 7. The display device as recited in claim 4, wherein theeach of the pixels corresponds to four pairs of two differently alignedcells differently aligned cells, one for amplitude modulation and theother for phase modulation.
 8. The display device as recited in claim 1,wherein the expanded image is generated to the first frame and thesecond frame of image by: writing each pixel value in the input imageinto a set of 2×2 sub-pixels simultaneously addressable by a pair ofX-decoders and Y-decoders; and processing the expanded image to minimizevisual errors when the first and second frames are alternativelydisplayed at the twice refresh rate.
 9. The display device as recited inclaim 8, wherein intensities of the first frame and the intensities ofthe second frame are identical.
 10. The display device as recited inclaim 8, wherein intensities of the first frame and the intensities ofthe second frame are different but a sum of the intensities of the firstand second frames is substantially identical to the intensities of theinput image.
 11. The display device as recited in claim 1, wherein thesecond frame is produced from the first frame by separating the expandedimage based on intensities of the input image.
 12. The display device asrecited in claim 1, wherein the second frame is produced from the firstframe by shifting the first frame by one sub-pixel along a predefineddirection to generate the second frame.
 13. The display device asrecited in claim 12, wherein the predefined direction is diagonal,vertical or horizontal.
 14. The display device as recited in claim 13,further comprising a set of X-decoders and Y-decoders for accessing thememory array, and wherein the second frame is produced from the firstframe by controlling X-decoders and Y-decoders to address two lines andtwo columns of the pixel elements all the time.
 15. The display deviceas recited in claim 14, wherein each of the X-decoder and Y-decoder isdesigned to address two lines of sub-pixels at the same time, the eachof the X-decoder and Y-decoder is controlled by a switch signal toalternate a selection of two neighboring lines of sub-pixels.
 16. Thedisplay device as recited in claim 1, wherein the display device is partof a liquid crystal display (LCD), a projection system.